Systems are known which provide for connectivity in and among networks of computerized equipment. Such systems, ideally, must accommodate a proliferation of different network interfaces and hardware, as no single "standard" is adopted universally.
In order to permit computer systems to communicate, regardless of connection method or vendor-specific hardware implementation, or to permit different networks to communicate or be "internetworked", modularized/layered solutions have been proposed for the problems associated with interconnectivity. Layering divides the task of interconnection and communication into pieces (layers), wherein each layer solves a piece of the problem or provides a particular function and is interfaced to adjacent layers. Each of the layers is responsible for providing a service to ensure that the communication is properly effected. Examples of some services provided by the various layers are error detection, error recovery and routing among many communication paths. All the layers in conjunction present the overall communication solution. Experience has shown that modularizing in layers with well defined functional interfaces, divides and effectively reduces the complexity of the connectivity problem and leads to a more flexible and extensible solution.
A model for describing the layers in a network has been posited by the International Standards Organization (ISO). The ISO open systems interconnection (OSI) model is a seven-layer model, illustrated in FIG. 1, which provides a standard for describing a network and facilitating computer communications. OSI and other layered computer network communications standards are discussed in detail in Unix Network Programming by W. Richard Stevens, and Handbook of Computer-Communication Standards by William Stallings, which are incorporated herein by reference. The OSI model defines the layers and units of information that pass along a network. As illustrated, data from an application or process running on a first host (HOST A) moves down the model network layers to a Physical layer. The Physical layer defines the physical connection which transmits raw bits across a communication channel to another host (HOST B) and up corresponding layers to a process running thereon. OSI, while defining a model or framework in which standards and protocols can be developed at each layer, allows for a flexible approach for implementation of the model.
Layered protocols and interfaces therebetween have been defined, which provide specifications for communication between a process or program being executed on one computer's operating system and another process running on another computer. Transmission Control Protocol/Internet Protocol (TCP/IP) are two protocols that are part of a protocol suite or family of protocols layered and designed to connect computer systems that use different operating systems and network technologies. TCP/IP, which provides a common set of protocols for invocation on dissimilar interconnected systems, is illustrated and mapped in FIG. 1a to analogous layers of the OSI model.
TCP/IP is a four layer protocol suite which facilitates the interconnection of two or more computer systems on the same or different networks and in certain networks, such as the Internet, is a requirement for interoperability. The four layers, comprise two independent protocols: TCP which can be used to access applications on other systems within a single network; and IP which permits identification of source and destination addresses for communication between systems on different networks.
As illustrated in FIG. 2, application or process data communicated via TCP/IP is "packetized" as it passes down layers through the protocol suite. The original process data first has an information block called a TCP Header prefatorily appended thereto in a TCP layer, to form a TCP packet. The TCP Header contains information to assure that the data travels from point to point reliably without picking up errors or getting lost. An IP layer repacketizes the TCP packet into an IP packet, by adding an IP Header which contains information needed to get the packet to a destination node. The IP packet is further packetized, such as in ANSI/IEEE 802 local area network protocol, with an additional Logical Link Control (LLC) address header and a control header at an LLC layer, to form an LLC Protocol Data Unit (LLCPDU). The LLCPDU is "framed" for transmission by addition of a Media Access Control Header and Trailer, to form a MAC Frame for communication between two TCP/IP facilities.
It is apparent that a considerable amount of "baggage", in the form of headers and trailer, is added to data which is transmitted between facilities using a layered protocol suite, such as TCP/IP and other OSI modelled families. Many additional bits are added at the various layers and must be processed for ultimate transmission across a communication channel at the physical layer. At its destination, the transmitted frame must be unpacketized according to embedded instructions and passed upward through the layered protocols to its receiving application or process.
Aside from the significant processing overhead associated with packetizing data for network and internetwork transmission, data itself may be redundant, such that real costs are associated with putting data and all its protocol suite baggage across the communication channel.
Where the communication channel is integral to a wide area network (WAN) the impact of redundant data and protocol suite baggage on transmission costs can be measured in dollars. WAN links, such as dial-up phone lines, typically have relatively low, fixed upper limit data transmission rates and are billed based on connect time, packet count or bandwidth use. Thus, the extensive packetization and resultant bits that must be transmitted increase connect time and dollar cost. Additionally, the numerous appended bits significantly increase the possibility of transmission errors on less reliable lines. Where WAN links, such as analog leased lines, and Dataphone Digital Services (DDS) are used, monthly or annual subscription fees recur and to a great extent are consumed by traffic comprising the transmission of redundant bits and protocol attributable header and trailer information.
Compression schemes and apparatus are known which reduce the quantity of bits that must be transmitted across the communication channel. However, data compressing modems, such as disclosed in U.S. Pat. No. 4,748,639, compress data streams after packetization for transmission over the communication channel. Significant additional hardware is required, in the form of a compression modem and a decompression modem. The additional hardware, which translates into significantly increased system cost, requires frequency tables to recodify and decodify or decompress characters in a data stream in accordance with a compressed character code. Such a compression scheme is not host resident, does not take advantage of the modular/layered structure of the network communication system and lacks the flexibility to reduce the number of packets and headers generated in the transmission of a particular data stream. Further, because compression modems service multiple connections and data that may come from one or more systems, redundancy of data tends to be random. It is appreciated that randomly redundant data is more difficult to compress.
Network bridge products are available that also have data compression facilities. However, much like compression modems, bridges do not provide host resident end-to-end compression facilities. Bridges also do not have the capability to reduce packet count. Like compression modems, bridge products are not connection specific and must process data from various connections which tend to provide data that is randomly redundant. Thus data compression with bridge products may not achieve optimal compression ratios.
Host resident compression is available with application programs that incorporate compression algorithms. However, such programs typically retrieve a file from a mass storage device such as disk storage, process the file and return it to the disk. The processed file must then be retrieved from the disk to be shipped through the network layers for transmission. In addition to the significant additional overhead required for the disk storage and retrieval, such compression applications require that process data be effectively compiled as a compressed data file and recompiled or decompressed at a destination facility. Such compression and decompression significantly delays the availability of data subjected to this process of data transmission. Such decompression programs, although host resident, are typically not integrated with particular applications and must be invoked so that they are not transparent to the user.